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DETAILED ACTION 



1. This action is responsive to communications of amendment received 12/24/2003. 

2. The disposition of the claims is as follows: claims 1-27 are pending in the application. 
Claims 1,11 and 21 are independent claims. No claims have been amended. 



3. In view of the amended specification the objection to the disclosure is withdrawn. 



4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this 
or a foreign country, before the invention thereof by the applicant for a patent 

Claims 1-8, 11-18 and 21-25 are rejected under 35 U.S.C. 102(a) as being disclosed by 
DeLeeuw, (US Patent Number 6,353,450B1). 

A. Claim 1 , "A method for displaying resource aids in a display area [This may be assisted 
through the use of colored dots attached to the user's hands or fingers or through other computer 
vision methods. Optionally, distinctive colors may be placed on specific objects (like fingers and 
thumbs, for example, or other props), to help the computer vision components of embodiments 
of the present invention isolate and identify objects to be recognized as input event generators. - 



Specification 



Claim Rejections - 35 USC § 102 
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col. 4, Ins. 26-33 : so that the user may see both layers clearly and substantially simultaneously , 
and interact with user interface elements in the transparent layer. - col. 4, Ins. 47-49], said 
method comprising the steps of: displaying a user interface comprising at least one displayable 
object within a display area [the user interacts with display objects - col. 4, In. 38]; and 
responsive to an initiating event, placing a transparent resource aid [supra: to help . . .; and user 
may see both . . .] within said display area in association with said at least one displayable object, 
such that said at least one displayable object is not obscured by said transparent resource aid [the 
image of the user is captured by the video camera and rendered in a transparent manner to the 
display . - col. 4, Ins. 36-38]" is disclosed in col. 4, Ins. 23-65. 

B. Claim 2, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the step of: responding to said initiating event, wherein said initiating 
event is at least one of a cursor placement, an occurrence of a user-defined event, and a user 
input [Many techniques and devices for communicating input data from a user to a computer 
system have been developed. Keyboards, cursor movement devices (such as a mouse, trackball, 
and joystick, for example), pens and tablets, bar code scanners, and other devices have all been 
used to direct a computer to perform selected tasks. - col. 1, Ins. 12-15; 

The input capability of some embodiments of the present invention may be provided 
without any modification to existing application programs. In embodiments of the present 
invention, these movements may be used to direct actions within the computer system such as 
grasping or selecting icons, transparent controls, and other application program elements, much 
as the user now does with a mouse or other selection device. - col. 3, Ins. 19-26 
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]" is disclosed supra for claim 1. Wherein user interaction with display program or transparent 
user interface elements corresponds to an occurrence of a user-defined event or a user input. 

C. Claim 3, "The method for displaying resource aid in accordance with claim 1, said 
method further comprising the step of: responding to said initiating event, wherein said initiating 
event is a position of a cursor over a sensitive region of said displayable object" is disclosed 
supra for claim 1 and in col. 3, Ins. 1 1-42. Wherein image of user's hands and fingers are 
sensitive areas for interacting with application program within the computer system such as 
grasping or selecting icons or transparent controls. 

D. Claim 4, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the steps of: determining a graphical output format for said 
transparent resource aid; adjusting a transparency of said transparent resource aid according to 
user transparency preferences; determining a display position for said transparent resource aid; 
and adjusting said transparency of said transparent resource aid according to said display 
position" is disclosed supra for claim 1 and in col. 4, Ins. 62-65, col. 9, Ins. 32-46 and col. 18, 
Ins. 24-29. 

E. Claim 5, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the steps of: monitoring performance of a plurality of parts of a 
computer system; and compiling information for said transparent resource aid from said 
monitored performance for a selection from among said plurality of parts" is disclosed supra for 
claim 1 and in col. 3, Ins. 1 1-42. Wherein image of user's hands and fingers correspond to 
sensitive areas for interacting with application program within the computer system such as 
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grasping or selecting icons or transparent controls and also correspond to monitor performance 
of a plurality of parts. 

F. Claim 6, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the steps of: monitoring [This physical movement activity to create a 
desired result while observing the user's reflected image - col. 3, Ins. 26-28; providing a 
transparent layer of display data signals (such as video data signals communicated by a video 
camera, for example) over the top of another layer of display data signals on a computer display 
so that the user may see both layers clearly and substantially simultaneously, and interact with 
user interface elements in the transparent layer. - col. 4, Ins. 43-48] a plurality of transparency 
settings [For example, the desired level of opacity may be set for future video data signal 
processing. - col. 14, Ins. 42-43; One embodiment comprises a method for producing 
transparent computer graphics layers - col. 4, Ins. 54-56] for each of a plurality of displayable 
objects [the user interacts with display objects - col. 4, In. 38] displayed within said user 
interface; and compiling information [] for said transparency resource aid [user's reflected image 
- col. 3, In. 28] from said monitored transparency settings" is disclosed supra for claim 1 and in 
col. 3, Ins. 11-42; col. 4, Ins. 52-58; and col. 14, Ins. 40-45. 

G. Claim 7, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the step of: placing said transparent resource aid to maximize space 
remaining in said display area" is disclosed supra for claim 1 and in col. 4, Ins. 52-52. 

H. Claim 8, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the step of: placing said transparent resource aid, wherein said 
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transparent resource aid comprises at least one from among text, graphics, video, and audio" is 
disclosed supra for claim 1 and in col. 18, Ins. 37-47. 

I. Per independent claims 1 1 and 21, these are directed to a system and program, 
respectively, for performing the method of independent claim 1, and therefore are rejected to 
independent claim 1 . 

J. Per dependent claims 12-18, these are directed to a system for performing the method of 
dependent claims 2-8, respectively, and therefore are rejected to dependent claims 2-8. 
K. Per dependent claims 22-25, these are directed to a program for performing the method of 
dependent claims 4-7, respectively, and therefore are rejected to dependent claims 4-7. 



Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 10, 20 and 27 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
DeLeeuw, (US Patent Number 6,353,450B1), as applied to claims 1,11 and 21, respectively 
above, and further in view of Bagnas, (US Patent Number 5,805,163). 

A. Claim 10, "The method for displaying resource aids in accordance with claim 1, said 
method further comprising the step of: placing said transparent resource aid in a background of 
said display area in association with said at least one displayable object, wherein said transparent 
resource aid is darkened to draw attention to said at least one displayable object" is disclosed 
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supra for claim 1 and furthermore in col. 6, Ins. 21-32. However DeLeeuw does not appear to 
disclose "placing said transparent resource aid in a background of said display area in association 
with said at least one displayable object, wherein said transparent resource aid is darkened to 
draw attention to said at least one displayable object", but Bagnas does in col. 3, Ins. 49-62. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply "placing and monitoring transparent user interface elements in a 
live video stream as a method for user input" in combination with "darkened transparent window 
overlapping an opaque window" disclosed by Bagnas, and motivated to combine the teachings 
because it would provide for a need for transparent windows and controls in window 
environments as revealed by Bagnas in col. 1, lines 55-65. 

B. Per dependent claims 20 and 27, these are directed to a system and program, respectively, 
for performing the method of dependent claim 10 and therefore are rejected to dependent claim 
10. 

Allowable Subject Matter 

7. Claims 9, 19 and 26 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

Response to Arguments 

8. In view of arguments the following details are given to specifically point out the 
associations between given references with corresponding claims elements. 
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A. With regard to [reference DeLeeuw] and claim 1 , [may be assisted through . . . ; to help 
. . . ; so user may see both layers clearly and substantially simultaneously . . .] correspond to 
"resource aids"; [user image rendered in a transparent manner to the display] corresponds to 
"transparent resource aid within display area"; [The user then interacts with display objects] 
corresponds to "in association with said at least one displayable object". 

B. With regard to [reference DeLeeuw] and claim 2, [The user then interacts with display 
objects ... for example to provide input signals and events to the system] corresponds to both 
"initiating event is at least one of an occurrence of a user-defined event, and a user input" 

C. With regard to claim 3 and [reference DeLeeuw] given supra, claim 3 does not claim 
"sensitive region of said displayable object of the present invention is a region within each 
displayable object that when selected by user input, such as with a mouse, an initiating event is 
triggered". Even still DeLeeuw is sufficiently clear for [these movements may be used to direct 
actions within the computer system such as grasping or selecting icons, transparent controls, and 
other application program elements, much as the user now does with a mouse or other selection 
device. This physical movement activity to create a desired result while. . .] such that objects and 
icons are sufficiently sensitive enough to be grasped or selected by these movements of the 
user's reflected image creating a desired result "an initiating even is triggered". 

D. With regard to claim 5 and [reference DeLeeuw] given supra, [ This physical movement 
activity to create a desired result while observing the user's reflected image is very natural and 
intuitive for users. For example, users frequently look at reflected images of themselves in a 
mirror, for example, while manipulating objects with their hands without thinking very much 
about it. Additionally, it is typically easy for users to use either a dominant or non-dominant 
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hand or both hands to perform operations while viewing their reflected image.] is just as 
equivalent to "monitoring performance of a plurality of parts of a computer system" as observing 
or monitoring the results of mouse movement or mouse clicking on objects or icons. Both user's 
reflected image movement input and mouse movement input inherently involve movement 
tracking, speed and proper direction performance, response delay. This inherently gives a rough 
measure of movement tracking, speed, proper direction performance, and response delay. For 
example if you move left and the image goes right, directional movement is 180° out of phase; if 
you move quick and there is delay in the movement then there may be interrupts processing, very 
slow clock rate, etc. Thus [physical movement activity to create a desired result while observing 
the user's reflected imagejjnherently gives a rough measure of movement tracking, speed, proper 
direction performance, and response delay, all inherent in a computer system. 
E. With regard to claim 6 and [reference DeLeeuw] given supra, "monitoring a plurality of 
transparency settings for each of a plurality of displayable objects displayed within said user 
interface; and compiling information for said transparency resource aid from said monitored 
transparency settings" also corresponds with [In one embodiment, the user interacts with 
transparent user interface elements such as controls , for example, rendered transparently on the 
display along with the user's reflected image. In this embodiment, a video camera captures a live 
video stream of a scene which includes a user. A video capture component produces a reflected 
video stream, based on the captured video stream, which is capable of being manipulated by an 
operating system or other components of the system. The reflected video stream may be 
augmented by placing user interface elements , such as buttons and slider bars, for example, in a 
transparent manner , in the frames of the video stream at selected locations . The augmented and 
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reflected video stream may be displayed to the user so that the user may see himself or herself 
and the transparent user interface elements on the display, in addition to the existing display 
components . The user may then make the transparent video image of his or her hand or another 
object or prop, for example, move over or at least partially coincide with a transparent user 
interface element . Herein, "over or at least partially coinciding with" means that the 
representation of the object on the display at least partially occupies pixels used to represent the 
transparent user interface element on the display. This action may be detected by a component 
analyzing the video stream and may be used to enact changes to the element's appearance to 
indicate an activation or selection of the element . Element activation in this manner may also be 
used to send control messages to the operating system or an application program. - col. 3, In. 43 
- col. 4, In. 3]. Wherein [while observing the user's reflected image] user collects "compiling" 
sufficient information on [transparent interface elements, selected locations, existing display 
components] for element activation. 

F. Claims 1 1 and 21 disclosed as detailed supra for claim 1. 

G. Claims 12-18 disclosed and detailed supra for claims 12-18. 

H. Claims 22-25 disclosed and detailed supra for claims 4-7. 

I. With regard to claim 10 and [reference DeLeeuw] given supra, "placing said transparent 
resource aid in a background of said display area in association with said at least one displayable 
object, wherein said transparent resource aid is darkened to draw attention to said at least one 
displayable object" 

Given how DeLeeuw discloses "transparent resource aid" [user's reflected image and 
transparent elements and controls] and [allowing transparent display components overlaying 
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background windows - col. 6, Ins. 21-32] and how Bagnas discloses darkening of a transparent 
window overlaying a background window, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to apply user's reflected image and transparent 
elements and controls and allowing transparent display components overlaying background 
windows disclosed by DeLeeuw in combination with darkening of a transparent window 
overlaying a background window disclosed by Bagnas, and motivated to combine the teachings 
because it would just yield a darkened opacity for the desired level of opacity as revealed in col. 
14, lines 42-43 by DeLeeuw. 

J. Claims 20 and 27 disclosed and detailed supra for claim 10. 

Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 
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Responses 



10. Responses to this action should be mailed to: Commissioner of Patents and Trademarks, 
Washington, D.C. 20231. If applicant desires to fax a response, (703) 872-9314 may be used for 
formal communications. 

Please label "PROPOSED" or "DRAFT" for informal facsimile communications. Hand- 
delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, Arlington, VA., 
Sixth Floor (Receptionist). 



1 1 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Greg Cunningham whose telephone number is (703) 308-6109. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella, can be reached on (703) 308-6829. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-4700. 



Inquiries 





gfc 



MATTHEW C. BELLA 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 
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